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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 



Period for R ply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) E3 Claim(s) 1-35 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [3 Claim(s) 1-35 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) K The specification is objected to by the Examiner. 

10)£<] The drawing(s) filed on 20 December 2001 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 1 )□ The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)DAII b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 13 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 

1 ) [3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) d Notice of Informal Patent Application (PTO-152) 

3) £3 Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 
Specification 

1 . Applicant is reminded of the proper language and format for an abstract of the disclosure. 

The abstract should be in narrative form and generally limited to a single paragraph on a 
separate sheet within the range of 50 to 1 50 words. It is important that the abstract not exceed 
1 50 words in length since the space provided for the abstract on the computer tape used by the 
printer is limited. The form and legal phraseology often used in patent claims, such as "means" 
and "said," should be avoided. The abstract should describe the disclosure sufficiently to assist 
readers in deciding whether there is a need for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information given in the 
title. It should avoid using phrases which can be implied, such as, "The disclosure concerns," 
"The disclosure defined by this invention," "The disclosure describes," etc. 

2. The abstract of the disclosure is objected to because the phrase 'This invention 
comprises.. can be implied. Correction is required. See MPEP § 608.01(b). 



Claim Objections 

3. Claim 16 recites the limitation 'The method of Claim 16" and "the dub audio track radar 
measurements" in lines 1-2. There is insufficient antecedent basis for these limitations in the 
claim. Claim 16 depends from claim 16. 



Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 
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5. Claims 1-35 are rejected under 35 U.S.C. 102(b) as being anticipated by Bregler (US 
5,880,788). 

Regarding claims 1,15-17, 25-33 and 35, Bregler discloses "analyzing the original audio 
track to convert it into phonemes as a time-coded phoneme stream to identify corresponding 
visual facial motions of the original speaker to create frames of facial motion corresponding to 
speech phoneme utterance states and transformations (column 2, lines 28-36), storing these 
frames in a database (column 2, line 36), analyzing the second audio dub track to convert it to 
phonemes as a time-coded phoneme stream, using the second audio dub track time-coded 
phoneme stream to animate the original speaker's face (column 11, lines 9-12), synchronized 
to the second audio dub track to create natural continuous facial speech expression by 
the original speaker of the second dub audio track (column 1, 48-50). 

Regarding claim 2, Bregler discloses "wherein said second audio dub track is spoken in 
a language different from that of the original speaker." (column 1, lines 32-33) 

Regarding claim 3, Bregler discloses "wherein said phonemes comprise diphones." 
(column 4, line 41) 

Regarding claim 4, Bregler discloses "wherein said phonemes comprise triphones." 
(column 4, line 41) 

Regarding claim 5, Bregler discloses "using a set of fixed facial reference points to track 
the facial transformation from one phoneme to another phoneme." (column 1, lines 48-51) 

Regarding claims 6 and 19, Bregler discloses "using a computer motion tracking system 
to record the optical flow path for each fixed reference point." (column 1 1 , line 4) 
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Regarding claim 7, Bregler discloses "using a set of fixed facial reference points to track 
the facial transformation from one phoneme to another phoneme." (column 1, line 14) 

Regarding claim 8, Bregler discloses "accumulating a database of recorded triphone and 
diphone mouth transformation optical flow paths." (column 2, line 36; column 4, line 41) 

Regarding claim 9, Bregler discloses "adding an emotional elicitation process by using a 
computer motion tracking system to record a number of facial control points for each emotion." 
(column 1 1, line 4) 

Regarding claims 10, 20, 21 and 34, Bregler discloses wherein "the facial control points 
comprise the chin, outside of the mouth and inside of the lips." (column 1, line 49) 

Regarding claims 1 1 and 22, Bregler discloses "accumulating a database of visemes 
by recording the facial control points corresponding to each phoneme." (column 2, line 36) 

Regarding claim 12, Bregler discloses ''accumulating a database of muzzle patches by 
mapping the facial control points for each viseme." (column 2, line 36) 

Regarding claim 13, 14, 23 and 24, Bregler discloses "selecting muzzle patches from 
the speaker's viseme database based on the second dub audio track and phoneme to 
viseme sequence and applying the selected muzzle patches onto a three dimensional 
facial muzzle model." (column 2, lines 35-45) 

Regarding claim 18, Bregler discloses "in which both audio tracks are time stamped to 
frames to create a database of individual frames for each phoneme." (column 2, lines 35-36) 
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Conclusion 



6. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Gasper, et al. (US 5,630,017), Yi (US 5,502,790), Chen (US 5,608,839), Rivers (US 
5,61 5,301) and Waters, et al. (US 5,657,426) each disclose a method / apparatus for modifying 
an audio visual recording originally produced with an original audio track of an original speaker, 
using a second audio dub track of a second speaker, to produce a new audio visual recording 
with synchronized audio to facial expressive speech of the second audio dub track spoken by the 
original speaker. 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Rodney Fuller whose telephone number is (703) 306-5641 . The 
examiner can normally be reached on Monday through Friday from 8:00 am to 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Russ Adams, can be reached on (703) 308-2847. 

Rodney Fuller 
Primary Examiner 




June 30, 2003 



